“Volume-Panorama-Imaging” refers to such a technique that splices a series of two-dimensional images into one continuous image, which are obtained through moving a probe within the same plane. Due to a relatively large quantity of processed data, this technique generally utilizes a high-speed processor or computer to reconstruct the single image, and becomes increasingly widely used in the field of data pick-up in which a relatively large object is sensed with a minor probe, such as the acquisition of a fingerprint with a tiny probe. Particularly in the field of medical ultrasonic imaging, there is a higher requirement and a wide demand for the technique for the purpose of the aided iatric diagnosis.
Taking an ultrasonic instrument for example, due to the harmless, convenient and reliable characteristics of ultrasonic, the ultrasonic perspective has become a common and auxiliary approach for the doctor to observe a part within a human body structure and diagnose its illness, and the doctor can obtain ultrasonic images of the corresponding part within the body by manipulating a probe disposed on the skin surface of the human body. However, the scanned area of a general probe is limited, resulting in a restriction on the size of the single ultrasonic image that the doctor can see. When the single ultrasonic image fails to display the panorama of the part, the doctor has to move the probe back and forth so as to observe different regions of the part. Thus, when a region to be detected is located beyond an image, it is impossible to utilize the general measurement function of ultrasonic to directly measure the size of the region. Up to the present, in order to solve the above limitation on the depth of field for the probe, a preferable approach is to splice a series of images obtained through a back-and-forth scan of the same plane by the doctor into one “elongated” ultrasonic image based upon the correlation between the images by means of the technique of volume-panorama imaging, which facilitates the measurement of a relatively large region.
The concept of image splicing in the above volume-panorama imaging has been proposed very early. During the development of the ultrasonic instrument from M-ultrasonic to B-ultrasonic of a single scan line, a series of scan lines is spliced so as to compose a two-dimensional image, and then the concept of the compound B-Scanner is proposed. That is, a B-ultrasonic probe is fixed on a mechanical arm which limits the movement of the probe within the same plane, current positional information of the probe is recoded in a real-time way, and the information is used to splice a series of images obtained during the movement of the probe into one image. The method with the compound B-Scanner is very inconvenient due to its use of the mechanical arm. Moreover, since the mechanical arm is not practical for the modern handhold ultrasonic probe, its development ends in the field of the image splicing. In the latest decade, various techniques of volume-panorama imaging have been continuously proposed with respect to the image splicing of the handhold ultrasonic probe.
Generally speaking, the volume-panorama imaging includes two portions of alignment and splicing. The alignment includes the steps of calculating a relative offset (offx, offy) and a relative rotation angle θ between images and combining the relative offset (offx, offy) and relative rotation angle θ into one transform coefficient (offx, offy, θ) that can simply determine a geometry transform between two images. The splicing refers to a procedure of combining the two images into one image through the transform coefficient. The procedure of the splicing is seldom described in many patents because the alignment is generally considered as the critical step for achieving a correct volume-panorama image.
SAD (Sum of Absolute Difference) is used as one of methods for calculating the offset. According to the method of SAD, firstly, one image is generally divided into a plurality of parts, and for each of the parts, a region corresponding to that part is selected in another image; secondly, a SAD value is calculated based upon respective positional data of the two corresponding parts and the position of the region with the smallest SAD value is the best matching position for that part. In this way, the offset is obtained through a calculation of the relative relation between the positions of the two corresponding parts. The method of SAD is also used in some techniques to calculate the rotation angle. Similar to the calculation of the offset, the image is rotated within a range of angles based upon a preset step interval; a similarity with respect to another image is calculated for each rotation with the method of SAD; and the rotation position with the smallest SAD value is a desired rotation angle.
In addition to the method of SAD, the method of MLS (Moving Least Square) can also be used to calculate the offset or the rotation angle. For example, the method of MLS is utilized in both patents of U.S. Pat. No. 5,566,674 owned by Siemens Corp. and of U.S. Pat. No. 6,605,042 B2 owned by GE Corp.
Hereinafter, an introduction will be made on the patent of U.S. Pat. No. 5,566,674 for Siemens Corp.
The method of the patent comprises the steps of:
a) dividing the n-th frame Fn into a plurality of regions;
b) calculating a local motion vector with the method of SAD;
c) correcting the local motion vector through a fuzzy logic to obtain a final local motion vector;
d) calculating a global motion vector and a rotation angle with the method of MLS, i.e. calculating a transform coefficient;
e) splicing the Fn to a volume-panorama image in which previous n−1 frames are spliced together so as to generate a new spliced image;
f) sharpening the image; and
g) n=n+1 and returning to the step of a) if the n-th frame exists, otherwise outputting the spliced image as a resultant image.
Specifically, the n-th image frame is divided into a plurality of small image regions, wherein the i-th sub-region is referred as n(i); a region to be searched is determined on a current spliced image; the n(i) is moved within the searched region; SAD value is calculated based upon data of each point within a corresponding position during the movement; and the position of the MSAD (Minimum SAD, i.e. the smallest SAD value) is judged as the best match for the n(i), and the offset between that position and the original position of the region n(i) is the local offset vector of the region. The regional SAD is calculated as follows:
      SAD          m      ,      n        =            ∑              i        =        1            l        ⁢                  ∑                  j          =          1                k            ⁢                                            X                          i              ,              j                                -                      Y                                          i                +                m                            ,                              j                +                n                                                                
Where, l and k define the size of the region n(i), X and Y represent the grayscale values of a point within the region n(i) and of a corresponding point within the searched region respectively, and m and n are the abscissa and the ordinate of an arbitrary point within the searched region. Thus, the point (m, n) corresponding which the SADm,n is the smallest is the best match for the corresponding n-th region n(i) within the searched region on the current spliced image, thereby determining the local offset vector v(i) of the region n(i).
In order to ensure the correctness of the local offset vector, the fuzzy logic is used in the step of c) to correct the vector. Specifically, two parameters are input to the fuzzy logic so as to evaluate the correctness of the current local offset vector and output a weighted value; and a weighted average is performed on the local offset value and a historical offset vector based upon the weighted value. Where, the first parameter is a difference between the MASD and the averaged SAD for judging the reliability of the MSAD, and the second parameter is a deviation of the current offset vector v(i) from a previous offset vector h(i). The final local offset is obtained through the correction for each initial local offset. Such a step of correcting the vector is absent in the patent of U.S. Pat. No. 6,605,042 B2 owned by GE Corp.
The type of a function for the mathematic model F(x) used in practice is often related to the physical background for the experiment and the actual distribution of data, typically including some parameters to be determined. The method of MLS is an important tool to estimate parameters depending upon experimental data, which estimates an optimum value for the parameters in the F(x) based on a set of discrete data obtained experimentally, and the optimum value allows a minimum of the total error between the model and the data obtained actually. The method of MLS has been used in both of the above patents, in virtue of which the global offset and rotation angle are calculated with the local offset.
During the splicing stage (the step of e), the weighted average is used to calculate grayscale values of an overlapped part according to U.S. Pat. No. 5,566,674. The weighted calculation is as follows:SC(n)=((1−m)·(SC(n−1))+(m·Input(n))
Where, SC(n−1) is a volume-panorama image in which previous n−1 frames are spliced together, and Input(n) is the n-th image frame. The weight m depends on the serial number of the image and the scan speed.
Due to the presence of the repeated weighting procedure during the splicing, the image need to be sharpened once after the splicing, that is, the spliced image need to be filtered in a high-pass way.
The drawbacks of the above prior art will be described mainly from the following aspects:
a) With regard to the aspect of calculating the rotation angle between the images with the method of SAD: this method requires a range of rotation and a step interval for each rotation angle to be determined in advance, but the step interval for the angle is difficult to determine, in that the quantity of the calculation will increase and the speed will be slowed down if the interval is too small, and the accuracy of the calculation will be lowered if the interval is too large, for an accurate rotation angle may be between two calculation angles.
b) With regard to the aspect of performing alignment with the spliced image: theoretically, the two immediately adjacent images are the most correlated. Since grayscale values of pixels on the spliced image are generally subject to a certain processing, the accuracy of the alignment result with the spliced image and the n-th image frame may be inferior to that of the alignment result with previous (n−1)-th image frame and the n-th image frame.
c) With regard to the aspect of performing alignment and splicing using each image frame among the series of images sequentially: taking the series of four image frames F1, F2, F3, F4 for example, the function D(Fi,Fj) represents an offset between two image frames, i.e. Fi and Fj, in X direction, which is obtained through a calculation with the MLS. It is assumed that D(F1,F2)=1.6, D(F2,F3)=3.8, D(F3, F4)=2.5, D(F1, F4)=8.5, and for the convenience of the description, it is assumed that an offset of an integer number is used for the splicing. In this case, due to a rounding, there will be an error of 1.1 pixels (i.e. 0.4+0.2+0.5) for a D(F1, F4) obtained through the calculation for every two frames. However, there is only an error of 0.5 pixel in the case of a direct splicing with the F1 and the F4. Consequently, such sequential processing is not advantageous for reducing the error. On the other hand, such sequential processing will lower the processing speed.
d) With regard to the aspect of calculating the offset with the SAD directly based upon data of each point within the regions divided from an image: in a case where the image has a large block of area in which the change of gradient is unobvious, this method will lower the accuracy of the calculation.
e) With regard to the aspect of calculating the transform coefficient with the based upon the local offset obtained with the SAD, particularly for the patent of U.S. Pat. No. 6,605,042 B2 for GE. Corp.: the experiments has proved that it is not the case that the offsets calculated from all the regions are correct due to noises of the ultrasonic images and organic movements, and consequently the transform coefficient may not be correct and result in a serious abnormity on the spliced image.